AHFD Aviation Human Factors Division Institute of Aviation University of Illinois at Urbana - Champaign 1 Airport Road Savoy , Illinois 61874 Imperfect Automation in Unmanned Aerial Vehicle Flight Control

We report a study that examines performance of pilots flying a simulated UAV in a highfidelity environment while using different types of perfect and imperfect automation aids. Fiftysix student pilots carried out simulated UAV missions containing a 3D view of terrain, a 2D topdown navigational map, system parameter gauges, and a message box. Trajectory and manipulation of a 3D camera was controlled by a multi-axis joystick. The pilots reported enemy activity and placement at specific coordinates, monitored UAV system parameters and monitored the terrain beneath their flight path for unexpected ground targets. These missions were carried out under seven different conditions: one baseline condition with no automation support; one condition with perfectly reliable auto-alerts (i.e., auditory alarms) for system failures, two conditions with imperfect auto-alerts for system failures (one condition with added false alarms, and the other with added misses; both at 67% reliability), one condition with a perfect auto-pilot flight control, one mission with an imperfect auto-pilot (at 67% reliability), and one condition with both perfectly reliable auto-alerts and a perfectly reliable auto-pilot flight control. The results of the study suggest that perfectly reliable auto-alerts and a perfectly reliable auto-pilot flight control both produce significant benefits to overall performance relative to baseline, with somewhat greater benefits realized by the auto-pilot. The imperfect auto-pilot flight control also provided many benefits relative to baseline, both to the flight control task and to concurrent tasks. The level of performance was often on par with the perfectly reliable autopiloting system, with pilots calibrating their trust in the imperfect auto-pilot quite accurately. In contrast, the imperfect auto-alerts provided almost no benefit to overall performance relative to baseline, and far worse than the perfect auto-alert aids, particularly during high workload situations. Pilots were unable to accurately calibrate their trust in the auto-alerts, and rated the reliability of the aid as much worse than it really was. From these results, we conclude that imperfections in the early stage automation aids (auto-alerts) harmed performance more than imperfections in the later stage (auto-pilot) aid. Furthermore, both types of imperfect auto-alerts (false alarms and misses) appeared to perform equally poorly, with each type damaging both primary and concurrent task performance relative to baseline, although in qualitatively different respects.